

SYSTEM FOR ASSISTING THE OPERATOR OF A TECHNICAL APPLIANCE 

The invention concerns a system for assisting an operator of a 
technical appliance, with an image display device, of which the 
5 image is provided within the normal viewing field of the 
operator, for indication of operationally relevant information in 
the form of images or symbols. 

Modern warning systems are being designed to warn the automobile 
10 operators of dangerous situations, of which the recognition has 
hitherto been the sole responsibility of the driver. For 
example, a "curve warner" provides the operator with early 
warning of the encountering of curves, which are too tight for 
the actual driving speed. Presently, this warning occurs using 
15 acoustic enunciators, for example "sharp left curve ahead," 
L-M and/or by offering optical warning signals in the dashboard or 

i't reflected into the windshield. 

;:?! This type of warning system is, as a rule, so designed, that the 

: LI 

;; 20 warning of a danger only occurs when it is to be presumed that 

! ;f the driver has not himself recognized this danger, since 

\2 otherwise the acceptance and the effect of the warning 

ffl information would be ignored or lost. In many warning systems, 

.s as. 

S it is however difficult or even impossible to determine or fix a 

25 suitable time point for generating the warning. On the one hand, 
the warning must occur early enough so that the driver can be 
appraised of the dangerous situation, should he not have 
recognized it himself. On the other hand, the warning should not 
occur so early, that a driver, who has already recognized the 
30 danger and intends to take appropriate measures, is unnecessarily 
disturbed or distracted. These two requirements can frequently 
not be accommodated at the same time. 

The same dilemma occurs in warning or information systems for 
35 other technical apparatus other than automotive vehicles. For 
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example, the operator of a machine or industrial apparatus is 
normally optically or acoustically warned or informed when 
particular situations occur which might possibly necessitate a 
rapid intervention. Too early or, as the case may be, too many 
5 warning signals could lead thereto that the attentiveness of the 
operator is reduced with time and important information is 
disregarded. On the other hand, the warning should not occur so 
late, that the operator might possibly not be able to react. 
^ < v 
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The invention is accomplished with providing the optical 
information for such a short period of time, that it is not 
15 consciously recognized, however is registered by the person. 
This phenomenon, referred to as "visual priming, 11 is described 
for example by A. J. Marcel in Conscious and unconscious 

perception: Experiments on visual masking and word recognition, 

Cognitive Psychology 15 (1983), Pages 197-237. The best known 
20 example of this phenomenon is the exchange of individual frames 
of a film sequence with frames having a completely different 
content, which the observer does not consciously comprehend 
during watching of the film, but which however do affect his 
perceptions or, as the case may be, his behavior. 

25 

This effect is used in the invention for sub-threshold warning, 
informing, or influencing of the operator of a machine. This 
makes possible for example a warning without distracting signals 
such as the illumination of lights or warning noises with 
30 conventional warning systems. 

During experiments for carrying out the invention, it has been 
demonstrated that the sub-threshold offering of operationally 
relevant information in an image display, of which the image is 
35 in the normal visual field of the operator, effectively 
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sensitizes the operator to the offered information. Thereby, the 
reaction time of the operator is substantially reduced when the 
situation, to which he was unconsciously prepared, actually 
occurs. The operator is however not disturbed or distracted by 
5 the information offered on the sub-threshold level. Thus, the 
operationally relevant information can immediately be displayed 
for a short period of time after the appropriate situation has 
been recognized, that is, without consideration as to whether the 
operator has initiated any manipulations. 

10 

According to the invention, an image or symbol, which reflects 
the operationally relevant information, is displayed at once for 
a short period of time, wherein the duration of the display is 
preferably below that of conscious recognition and above that of 

15 subconscious recognition of the operator. Therein, the threshold 
of subconscious recognition is defined as the display duration 
with which the information is just barely accepted unconsciously, 
that is, has an effect on the later behavior of the operator, and 
the conscious recognition threshold is the duration of display at 

20 which the operator becomes conscious of the information. 

These two recognition thresholds can, by the way, be influenced 
by the size and magnitude or intensity of the displayed image 
and, besides this, by whether the image or symbol is offered in 
25 the center of the field of vision or at the edge. It has been 
shown that a short time presentation in the periphery of the 
visual field influences the cognition or recognition or, as the 
case may be, the behavior of the operator. 

30 Basis on the condition that the operationally relevant 
information could be offered even in the periphery of the field 
of vision of the operator, the invention is suitable also for 
technical appliances, such as for example, an automobile. 

35 When adapting the invention to an automobile, one employs 
normally one or more devices for automatic recognition of 
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situations, which may possibly require a reaction of the 
automobile driver, and a device for controlling the image display 
device, in order to display the appropriate operationally 
relevant information as soon as the appropriate situation is 
5 recognized. 

For recognition of dangerous situations, there can be employed 
for example devices for recognition of an inappropriate speed of 
the vehicle prior to a curve, for recognition of traffic signs, 
-10 for recognition of road conditions, for recognition of traffic 
signals, for recognition of approach situations, for recognition 
of crossing situations, for recognition of pedestrians and/or for 
recognition of low stimulation or signal intensity and tedious 
driving situations. 

15 

The invention is suitable not only for sensitization with respect 
to dangerous situations, but rather also to sensitize automobile 
drivers or operators of other technical apparatus to various 
other important conditions without imposing or disturbing, i.e. 

20 conditions which might possibly require a reaction. Devices for 
recognition of various possible important situations during the 
operation of a vehicle, which can advantageously be combined or 
utilized with the present invention, are for example a navigation 
device and/or a device for monitoring of operational values of 

25 the vehicle, for example the status of the fuel. 

The recognition signals of the above-mentioned devices can in 
many ways be combined with each other, in order to prepare the 
information and in present it in an optimal manner to the driver. 

30 For example one symbol may be a symbol, which alerts the driver 
below the threshold of recognition to the possible need of a fuel 
stop, and can be given exactly then when the navigation system 
determines that a filling station is being approached. The 
briefly displayed images or symbols can be pictures of 

35 corresponding situations, pictograms, traffic symbols, arrows or 
written words . 
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In a preferred embodiment, the image display device is a so- 
called heads-up display. In an automobile, one realizes a heads- 
up display thereby that the image information is reflected into 
5 the visual field of the driver via the windshield. The reflected 
image information can in a first stage or level be the 
conventional operating parameters such as RPM, speed, blinker, 
etc. and the operationally relevant information according to the 
invention can supplementally be reflected or reflected during an 
-10 appropriate short time fade-out of the main image. 

The invention is also suitable for retrofitting to vehicles, 
which have for example conventional individual instruments on the 
instrument panel. In such a case, it can be useful when all the 
15 short time images are reflected via the windshield into the 
visual field of the driver. 

For adapting a motorcycle or other machine or vehicle, which has 
no suitable reflecting device such as a windshield, a heads-up 
20 display can be designed in the manner of, for example, glasses or 
goggles wearable by the operator. 

In place of a heads-up display, the image display device can be a 
pixel based controllable display screen for direct viewing by the 

25 operator, for example an LCD-display on the instrument panel, 
upon which important operational parameters of the vehicle are 
represented. As already described, the briefly offered images, 
even when peripherally displayed, leave behind a measurable 
impression with the driver, so that it is not detrimental when he 

30 normally does not look directly at the image screen or, as the 
case may be, in complex driving situations frequently changes his 
direction of viewing. 

For safety, the short time images can also be offered multiple 
35 times sequentially at spaced-apart time intervals. This is 
possible, since in accordance with the invention it is not 
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necessary to wait to see whether the driver reacts by himself. 
Rather, all necessary warning or other information can be given 
immediately, so that substantially more time latitude exists than 
with conventional warning systems. 



It is useful when the duration, with which the respective image 
or symbol is displayed for a short period of time, can be 
adjusted or controlled by the operator. Thereby, the operator 
can adapt the system precisely to his personal threshold of 
10 recognition. In order to facilitate this, the system can offer 
an appropriate adjustment procedure, through which the operator 



Further characteristics and advantages of the invention can be 



15 seen from the following description of embodiments and from the 



nj Fig. 1 - a flow diagram for illustrating the process steps in a 

! ! system for assisting the operator of a technical appliance using 

20 sub-threshold display of information, and 

: Fig. 2 - a test apparatus for determining the effectiveness of 

j the invention. . - 

25 A system for assisting a vehicle operator includes for example a 
device for recognition of an inappropriate speed of the vehicle 
prior to a curve, a device for recognition of traffic signs, a 
device for recognition of the road conditions, a device for 
recognition of traffic signals, a device for recognition of 

30 approach situations, a device for recognition of crossing 
situations, a device for recognition for pedestrians and/or a 
device for recognition of difficult to recognize and tedious 
driving situations. 

35 Besides this, there is a freely or variably programmable display 
device in the vehicle, for example an LCD-display in the 



5 



is guided. 



figures, to which reference is made. There is shown: 
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instrument panel, on which the operating parameters such as for 
example, speed, RPM, etc. are displayed. Alternatively, the 
display device can be a heads -up display. 

5 As shown in Fig. 1, upon recognition of a particular environment 
or situation (SI) the recognized situation is evaluated (S2) with 
respect to its danger potential or, as the case may be, its 
importance. Subsequently, there occurs a selection of a suitable 
excitation or stimulation based on type and duration (S3) , 

-10 wherein the appropriate images or symbols and the appropriate 
display duration can be stored, for example in a storage chip or 
component. Subsequently, there occurs the stimulus (S4) , in 
which the image or symbol is briefly displayed on the display 
device. These process steps can be carried out or, as the case 

15 may be, controlled by a suitably programmed microprocessor. 

The respective display durations are determined in advance for 
example by testing. The display duration is selected to be so 
short, that the appropriate image or symbol is just below the 
20 threshold level of conscious recognition of the driver. 

Test Device 1 

A test vehicle with a variously controllable display screen on 
the instrument panel in place of the conventional individual 

25 instruments was so modified, that the reaction of the driver to 
traffic signs and traffic lights could be measured. The reaction 
time of a sufficiently large number of test persons was measured 
once with and once without subconscious optical sensitization. 
For sensitization of the test persons, symbols of the traffic 

30 signals or, as the case may be, traffic lights were displayed 
approximately 50 ms on the display screen, before the traffic 
signs or, as the case may be, traffic lights actually appeared in 
the visual field of the test person. With this subconscious 
sensitization, a significant reduction of the reaction time of 50 

35 ms in comparison to the reaction time without sensitization 
resulted. 
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Test Device 2 

A similar test vehicle with an image display screen displaying a 
combination of instruments was allowed to follow a preceding 
5 vehicle traveling at 20-40 km/h. The test person was required to 
operate the brake pedal as soon as the brake light of the lead 
vehicle was illuminated. Coincidentally, in half the cases, the 
image of two illuminated backlights (vehicle stoplights) was 
blended into the combination instrument, before the vehicle brake 
10 lights actually illuminated. The duration of the blending in of 
the display of the image was selected to be so short, that the 
driver did not consciously recognize the images. 

Fig. 2 shows the instrument panel of the test vehicle from the 
15 view of the driver at precisely the instant, in which the 
symbolic image 1 of two backlights appear in the combination 

instrument 2 outside the tachometer 3 . The preceding or lead 

vehicle is not shown in Fig. 2. In the cases with subconscious 
sensitization, a significant reduction of the reaction time of 
20 150 ms occurred in comparison to the cases without sensitization. 

The described test systems were designed especially for testing 
the effectiveness of the invention and do not represent a 
commercially suitable warning system. The person working in this 

25 art would, however, recognize that the use of many warning 
systems for vehicle drivers, which are presently available, can 
be substantially improved in accordance with the present 
invention. For example, if one associated the curve warner with 
the present invention in such a manner that the driver is 

30 presented with the clearly recognizable optical warning signal 
(for example the traffic symbol for "curve") mirrored in 
windshield or displayed on the instrument panel with a duration 
below the threshold of recognition. As a warning signal, it 
would also be possible to use the image of a typical curve with a 

35 radius or curvature as detected by the curve warner. 
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If one takes into the consideration the fact that 90% of all 
accidents of vehicles could be prevented if the driver had better 
control over the vehicle in the last 500 ms prior to the 
5 accident, then the invention makes it possible, to increase the 
attention of the driver in the decisive phases, without however 
unintentionally disturbing or distracting him. 

In comparison to conventional warning systems, the information 
.10 offered below the threshold of recognition can always be offered, 
even independent of whether or not the operator has actually 
recognized a dangerous situation. For example, in situations, in 
which the vehicle is urgently to be brought to a halt, for 
example at a red light or a stop sign, it is useful, to offer the 
15 warning information independent of whether or not the operator 
has actually taken action or not. 
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